Full-low evaluation methods for derivative-free optimization

We propose a new class of rigorous methods for derivative-free optimization with the aim delivering efficient and robust numerical performance functions all types, from smooth to non-smooth, under different noise regimes. To this end, we have developed methods, called Full-Low Evaluation organized around two main types iterations. The first iteration type (called Full-Eval) is expensive in function evaluations, but exhibits good non-noisy cases. For theory, consider line search based on an approximate gradient, backtracking until sufficient decrease condition satisfied. In practice, gradient was approximated via finite differences, direction calculated by quasi-Newton step (BFGS). second Low-Eval) cheap yet more presence or non-smoothness. direct search, practice use probabilistic one random its negative. A switch Full-Eval Low-Eval iterations values line-search direct-search stepsizes. If enough steps are taken, derive complexity result gradient-descent type. Under failure Full-Eval, become drivers convergence yielding non-smooth convergence. shown be across problems levels smoothness noise.